Control of Depth-Sensing Camera via Plane of Interaction

Depth-sensing cameras (e.g. Kinect or Creative Gesture Camera) are exploited in many computer vision and augmented reality applications. They can also serve as a key component of natural user interaction via virtual keyboard, body pose or hand gestures. We integrated both these functions proposing the “Plane of Interaction” (POI) which is a solid flat surface placed on the reference plane (table top, floor). Calibrated camera/projector system automatically identifies the position of POI surface, projects the virtual menu buttons onto it and recognizes which button was “clicked” by hand (fingertip). The proposed POI was tested with the camera/projector prototyping setup. POI allows natural and quite robust interaction in this specific environment.